{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Guaranteeing valid output syntax\n",
    "\n",
    "Large language models are great at generating useful outputs, but they are not great at guaranteeing that those outputs follow a specific format. This can cause problems when we want to use the outputs of a language model as input to another system. For example, if we want to use a language model to generate a JSON object, we need to make sure that the output is valid JSON. This can be a real pain with standard APIs, but with `guidance` we can both accelerate inference speed and ensure that generated JSON is always valid.\n",
    "\n",
    "This notebook shows how to generate a JSON object we know will have a valid format. The example used here is a generating a random character profile for a game, but the ideas are readily applicable to any scenario where you want JSON output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import guidance\n",
    "\n",
    "# define the model we will use\n",
    "lm = guidance.models.LlamaCpp(\"/home/scottlundberg_google_com/models/mistral-7b-v0.1.Q8_0.gguf\", n_gpu_layers=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style='margin: 0px; padding: 0px; padding-left: 8px; margin-left: -8px; border-radius: 0px; border-left: 1px solid rgba(127, 127, 127, 0.2); white-space: pre-wrap; font-family: ColfaxAI, Arial; font-size: 15px; line-height: 23px;'>{\n",
       "    &quot;description&quot; : &quot;A quick and nimble fighter&quot;,\n",
       "    &quot;name&quot; :<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> &quot;</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>F</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>ighter</span>&quot;,\n",
       "    &quot;age&quot; :<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> </span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>2</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>0</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>,</span>\n",
       "    &quot;armour&quot; :<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> &quot;</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>le</span>ather&quot;,\n",
       "    &quot;weapon&quot; :<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> &quot;</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>sw</span>ord&quot;,\n",
       "    &quot;class&quot; :<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> &quot;</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>fig</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>h</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>ter</span>&quot;,\n",
       "    &quot;mantra&quot; :<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> &quot;</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>I</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> am</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> a</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> fighter</span>&quot;,\n",
       "    &quot;strength&quot; :<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> </span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>1</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>0</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>,</span>\n",
       "    &quot;quest_items&quot; :<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'> [</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>&quot;</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>sw</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>ord</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>&quot;</span>, <span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>&quot;</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>sh</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>ield</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>&quot;</span>, <span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>&quot;</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>arm</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>our</span><span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>&quot;</span>]\n",
       "<span style='background-color: rgba(0.0, 165.0, 0, 0.15); border-radius: 3px;' title='1.0'>}</span></pre>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from guidance import gen, select\n",
    "\n",
    "# we can pre-define valid option sets\n",
    "sample_weapons = [\"sword\", \"axe\", \"mace\", \"spear\", \"bow\", \"crossbow\"]\n",
    "sample_armor = [\"leather\", \"chainmail\", \"plate\"]\n",
    "\n",
    "# define a re-usable \"guidance function\" that we can use below\n",
    "@guidance\n",
    "def quoted_list(lm, name, n):\n",
    "    for i in range(n):\n",
    "        if i > 0:\n",
    "            lm += \", \"\n",
    "        lm += '\"' + gen(name, list_append=True, stop='\"') + '\"'\n",
    "    return lm\n",
    "\n",
    "@guidance\n",
    "def generate_character(\n",
    "    lm,\n",
    "    character_one_liner,\n",
    "    weapons: list[str] = sample_weapons,\n",
    "    armour: list[str] = sample_armor,\n",
    "    n_items: int = 3\n",
    "):\n",
    "    lm += f'''\\\n",
    "    {{\n",
    "        \"description\" : \"{character_one_liner}\",\n",
    "        \"name\" : \"{gen(\"character_name\", stop='\"')}\",\n",
    "        \"age\" : {gen(\"age\", regex=\"[0-9]+\")},\n",
    "        \"armour\" : \"{select(armour, name=\"armor\")}\",\n",
    "        \"weapon\" : \"{select(weapons, name=\"weapon\")}\",\n",
    "        \"class\" : \"{gen(\"character_class\", stop='\"')}\",\n",
    "        \"mantra\" : \"{gen(\"mantra\", stop='\"')}\",\n",
    "        \"strength\" : {gen(\"age\", regex=\"[0-9]+\")},\n",
    "        \"quest_items\" : [{quoted_list(\"quest_items\", n_items)}]\n",
    "    }}'''\n",
    "    return lm\n",
    "\n",
    "\n",
    "generation = lm + generate_character(\"A quick and nimble fighter\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have produced valid JSON:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded json:\n",
      "{\n",
      "    \"description\": \"A quick and nimble fighter\",\n",
      "    \"name\": \"Fighter\",\n",
      "    \"age\": 20,\n",
      "    \"armour\": \"leather\",\n",
      "    \"weapon\": \"sword\",\n",
      "    \"class\": \"fighter\",\n",
      "    \"mantra\": \"I am a fighter\",\n",
      "    \"strength\": 10,\n",
      "    \"quest_items\": [\n",
      "        \"sword\",\n",
      "        \"shield\",\n",
      "        \"armour\"\n",
      "    ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "gen_json = json.loads(generation.__str__())\n",
    "\n",
    "print(f\"Loaded json:\\n{json.dumps(gen_json, indent=4)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have also captured our generated text and can access it like a dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'sword'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generation[\"weapon\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height: 1px; opacity: 0.5; border: none; background: #cccccc;\">\n",
    "<div style=\"text-align: center; opacity: 0.5\">Have an idea for more helpful examples? Pull requests that add to this documentation notebook are encouraged!</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
